News — 10 October, 2017
Export Tool | 03 Customise Map Features
This blog is the third in a series of ‘Learn Export Tool’ posts and focuses on how to customise the OSM map features for export. Following the launch of the revamped tool on the 18th September 2017, the first Learn blog focused on ‘Selecting Area of Interest’, the second focused on ‘Data File Formats’ and the last will look at examples of ‘Applying Exported Data’.
The Humanitarian OpenStreetMap Team (HOT) Export Tool allows users to create custom OpenStreetMap (OSM) extracts anywhere in the world, by selecting an area of interest, map features and file formats. Within minutes, up-to-date OSM data is exported, filtered and converted.
The tool allows the user to customise the data selected within the defined area of interest. The OSM data is defined by using tag filters and key selections, with the Tag Tree or YAML Form. The Tag Tree is for common use cases, presenting a curated set of filters and selections, where the YAML configuration provides complete control over filters and selections, using a SQL-like filter definition.
This blog is the third in a series of ‘Learn Export Tool’ posts following the launch of the revamped tool on the 18th September 2017. The first blog of the series was on ‘Selecting Area of Interest’ (September 25th), the second focused on ‘Data File Formats’ (October 2nd) and the last will look at examples of ‘Applying Exported Data’ (October 23rd).
Figure 1. Tag Tree
Figure 2. YAML Form
OSM is an open global database of tagged geographic features, with three types of elements:
Nodes, which represent a point on the surface of the earth
Ways, which are sets of nodes that can form lines or polygons
Relations, which are sets of nodes, ways or other relations
Each of these elements can have any number of key=value tags. For example, a post office may be represented by a way with the tags building=yes and amenity=post_office. Lets see how these tags can be defined in the Export Tool using the Tag Tree and YAML Form to filter OSM data.
Tag Tree
The Tag Tree is the simplest way to get started selecting features, simply by ticking the desired parent or child checkboxes. Please note that selecting a parent checkbox will add additional key=value tags associated to the theme, as well as the child checkboxes below it. Each parent checkbox has a different query to filter data, so it is highly recommended that the syntax for each theme is explored by hovering over the checkbox, which will provide an info box to the right.
Figure 3. Parent Checkbox and Filter Query
For example selecting the ‘Emergency’ parent checkbox, will automatically select the ‘Police Station’, ‘Ambulance Station’ and ‘Fire Station’ child checkboxes below it (Figure 3), yet in addition to this it will also select all tags where emergency=yes, amenity=police and amenity=fire_station in the following SQL query:
emergency IS NOT NULL OR amenity IN ('police','fire_station')
The parent checkboxes do not always represent what is available as children below, to try and account for all possible tags associated with a theme. OSM tags are continuously evolving and we want to ensure that the tool can adapt to these changes. Only the most commonly used key=value tags are represented as child checkboxes, and the parent checkboxes are used to try and encompass those not used as frequently, but fall within a theme. If you have suggestions for modifications to these themes and tags, please comment on the spreadsheet used to populate the Tag Tree, and we will try to incorporate it where appropriate (Figure 4).
Figure 4. Tag Tree Spreadsheet
YAML Form
Using a YAML configuration provides complete control over applying filters to the OSM data, by using a SQL-like filter definition to apply key=value tags. Please note that the Tag Tree also generates syntax on the YAML Form, so any parent and child checkboxes selected will automatically be applied in the YAML box (Figure 5). This acts as a starting point for the query which can be further edited.
Figure 5. YAML Configuration
The use of YAML was chosen due to its simplicity and compatibility with SQL. The YAML feature selection format is similar to style files used by programs such as osm2pgsql and imposm. It is whitespace sensitive, with each child element indented below its parent element, and preceded by a dash. This dash must have a space after it. Here is a basic example of a feature selection with 3 themes, buildings, waterways and hospitals:
YAML has Themes, and two data structures, Mapping and Lists
Theme in the above example is: buildings
Mappings in the above example are: types and select
Lists in the above example are: child elements of select and types
YAML: Themes
Themes are the top level keys in the YAML document, with valid characters including letters, numbers and underscores.
YAML: Geometry Types
The list values under the mappingtypes can be one or more of ‘- points’, ‘- lines’, ‘- polygons’. If the types key is omitted, all three geometry types will be included in the theme.
YAML: Column Selections
List items under the mappingselect key determines the columns for each theme. The following example will populate the ‘name’ and ‘amenity’ columns with their values from OSM:
YAML: Filters
Filters are under the where: key in each theme. They define what subset of OSM features belongs to that theme. The following example will filter the theme to only features where the key natural has the value waterway:
Please note It is almost always necessary to have some kind of filtering, otherwise the theme will simply include all OSM features for the given geometry types. A filter is specified using SQL-like syntax, with valid keywords IS NOT NULL, AND, OR, IN, =, !=.
JOSM Presets
Older versions of the Export Tool used JOSM Preset .XML files to define feature selections. The new version uses YAML as it is more flexible in how it transforms OSM data. The new Export Tool, however can help convert JOSM presets into YAML configurations, by selecting the ‘Load from JOSM Preset .XML’ button. Please note If the preset is more complex, it may need to be written as a new YAML configuration based on the ‘item’ elements contained in the XML.
Figure 6. Convert JOSM Preset
Configurations
YAML configurations can be defined and saved for future use through the ‘Configs’ page. It's useful to create one configuration for a project, which can then used on all exports related to that project. Give the configuration a ‘name’ and ‘description’ that will make it discoverable by other users. Unchecking the ‘Public’ checkbox will make the configuration visible to only the user.
Figure 7. Saved Configuration
Please note that configurations can be edited, which is useful for evolving a feature selection during the course of a project. Saved YAML configurations can be selected via the ‘Stored Configuration’ option on the ‘3 Data’ tab when creating an export. Use the Search bar to find configurations related to your project.
For further detailed information regarding feature selections and YAML, please visit the ‘Selecting Features’ and ‘YAML Specification’ in the ‘Learn’ section on the tool website.